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APPS Processing Steps 




Call Audio Feature Extraction 
to produce the Graphical 
Representation, given the 
audio file and the DFS 



Add symbolic form to DFS 
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Call Audio Feature Extraction 




Yes 


to produce the 






markers requested in the 






DFS, given the audio file 






and the DFS 





Add marker data to DFS 
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Call Audio Paper Formatting to 
produce the document type 
specified in DFS [output 
t] given DFSPayout*] 
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Audio Paper Formatting module 



input: APFS inciuding the result ^^402 
of feature extraction 





For each page in Layout[pages] 
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Format meta data in APFS as specified 
in Layout[meta data placement] 








Format APFS[Graphicai Representation] 
as specified in i ' /en the 
result of Audio Feature Extraction 








Generate bar codes as specified in Marker 
type, Maker frequency, and Marker n 
fields in the APFS 








Format markers as specified in the APFS 
given the bar codes 








Render the page given the formatted meta 
data, Graphical Representation, and 
markers 
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Bar code generation 

Inputs: 



' Bar code type (e.g., Interleaved 2 of 5), 

' No. of identifier digits in bar code, 502 

1 No. of time stamp digits in bar code, 

1 Time stamp vaiue 



Read Identifier field from APFS 



-504 



Convert Identifier to right-justified decimal string 506 




Convert time stamp value to right-justified decimal string 508 



Append right-justified time stamp (padded on tl 
left with zeros) to right-justified Identifier 



X 



Render bar code image of specified type containing 
Identifier + time stamp 

( Return ) 
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Document Format Specification 



Audio Feature Extraction 
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• Type = Musical recording 

• identifier = 1 768 



Audio amplitude extraction and graphical 
approximation. An svg file is output. "~ 



606 



61 0 Title = Locomotion 
61 2 Artist = John Coitrane 
61 4 Collection = Blue Train 
616--^ Publisher = Blue Note Records 
618^ Publication Date = 1 957 
620 ~ Begin time = 00:00:00 
622^ End time = 00:07:14 

Graphical Representation = Amplitude curve 624 

Marker type = bar code 628 

Marker frequency = 30 sec. intervals 630 

Layout type = One horizontal time line -"^ 632 
634 Layout pages = 1 

Layout marker placement = Above graphical representation 636 

Layout meta data placement = Centered at top of page 638 



FIG. 6a 
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Locomotion by John Coitrane 
on Blue Train — 
(c) 1957 Blue Note Records 
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Document Format Specification 



Audio Feature Extraction 



Type ~ Musical recording 
Identifier = 1769 
Title = Locomotion 
Artist = John Coiirane 
Collection = Blue Train 



Audio amplitude extraction and graphical^ 
approximation. An svg file is output, Musical 
Solo extraction is also applied, it outputs the 
beginning times and instrument for each 



musical solo, 



606 



Publisher = Blue Note Records 
Publication Date = 1 957 
Begin time = 00:00:00 
End time = 00:07:14 

Graphical Representation = Amplitude curve 
702 Feature extraction = Musical soios (output is instrument name and time when solo began} 
704 Marker type 1 = Instrument name above bar code above time stamp 
706 Marker type 2 = bar code 
708^^ Marker2 frequency = 0, 50%, 100% 

Layout type = One horizontal time line 

Layout pages - 1 

Layout marker 1 placement = Above graphical representation 710 
Layout marker 2 placement = Beiow time line 712 
Layout meta data placement = Centered at top of page 



FIG. 7a 
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Locomotion by John Coltrane 
on Blue Train — 
(c) 1957 Blue Note Records 
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IT 



602 



Document Format Specification 

Type = Radio program 
Identifier =1770 



Audio amplitude extraction and graphical 
approximation. An svg file is output. ^ 



Audio Feature Extraction 



Title = Fresh Air with host Terry Gross 
802 ^Annotation = Guest Bill O'Reilly 
Publication Date = Oct. 8, 2003 
Begin time = 00:00:00 
End time = 00:40:45 

Graphical Representation = Amplitude curve 
Marker type = bar code 
Marker frequency = regularly spaced 
Layout type = One horizontal time line 
Layout pages = 1 

Layout marker placement = Above graphical representation, 3-step staircase 
Layout meta data placement = Centered at top of page 



FIG. 8a 
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Fresh Air with Host Terry Gross 
Wed. Oct. 8, 2003 
Guest Bil! O'Reilly 
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104^ 

Document Format Specification 



Audio Feature Extraction 



Audio amplitude extraction and graphical 
approximation. An svg file is output, 



Type = Radio program 
Identifier = 1771 

Title = Fresh Air with host Terry Gros 
Annotation = Guest BIN O'Reilly 
Publication Date = Oct. 8, 2003 
Begin time = 00:00:00 
End time = 00:40:45 

Graphical Representation = Amplitude curve 
y Marker type = keywords, bar code, time stamp 

Marker placement = above time line 
^Marker frequency = user-defined 

- Marker 1 = (VVTC, bar code, 04:28, vert pos. 1} 

- Marker 2 = (NY Times, bar code, 09:08, vert. pos. 2) 

- Marker 3 = (Peabody, bar code, 12:30, vert. pos. 3) 

- Marker 11 = (People Mag, barcode, 39:10, vert, pos. 3) 
Layout type - One horizontal time line 

Layout pages - 1 

Layout marker placement = Above graphical representation, as provided 
Layout meta data placement = Centered at top of page 



FIG. 9a 



Fresh Air with Host Terry Gross 
Wed. Oct. 8, 2003 — 
Guest Bill O'Reilly 
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Document Format Specification 

Type = Radio program 
Identifier =1772 



Audio Feature Extraction 

Audio amplitude extraction and graphical 
approximation. An svg file is output. 
Speech recognition is also applied followed by 
matching to a given list of phrases. 



Title = Fresh Air with host Terry Gross 
Annotation = Guest Bill O'Reiiiy 

keyword search terms = "New York Terms" and 
lair and balanced" 
Publication Date = Oct. 8, 2003 
Begin time = 00:00:00 
End time = 00:40:45 

Graphical Representation = Amplitude curve 
y Feature extraction = speech recognition and keyword match to "New York Times" or 

"fair and balanced" 
' Marker type = matching search term, bar code, time stamp 

Marker placement = above time iine 
-'Marker frequency = user-defined 

^ — Marker 1 = ("fair and balanced", bar code, 02:31, vert pos. 2) 

Marker 2 = ("New York Times", bar code, 04:21 , vert pos. 1 ) 
Marker 3 = ("New York Times", bar code, 14:54, vert pos. 2) 

Marker 9 = ("New York Times", bar code, 35:12, vert pos. 3) 
Layout type = One horizontal time iine 
Layout pages = 1 

Layout marker placement = Above graphical representation, as provided 
Layout meta data placement = Centered at top of page 



FIG. 10a 




Fresh Air with Host Terry Gross 
Wed. Oct. 8, 2003 
Guest Bill O'Reilly 
keyword search terms = "New York Times" and "fair and balanced" 
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Document Format Specification 



Audio Feature Extraction 



Type = Radio program 
identifier =1773 



Audio amplitude extraction and graphical 

approximation. An svg file is output. 
Applause detection outputs time stamps. 



Title = A Prairie Home Companion 
Annotation = applause events shown 
Publication Date = Sun. Oct. 12, 2003 
Begin time = 00:00:00 
End time = 01 :58:56 

Graphical Representation = Amplitude curve 
702--^ Feature extraction = applause detection aipha~0.27 beta-1 .86 
Marker type = time stamp, bar code 
Marker placement = above time line 
Marker frequency = user-defined 

Marker 1 = (00:02:13, bar code, pos. 1) 

Marker 2 = (00:10:54, bar code, pos. 2) 

Marker 3 = (00:12:32, bar code, pos. 3) 



Layout type = One horizontal time Sine 
Layout pages = 1 

Layout marker placement = Above graphical representation, as provided 
Layout meta data placement = Centered at top of page 



Marker 1 6 = (01 :56:01 , bar code, pos. 2} 



FIG. 11a 
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A Prairie Home Companion 
Sun. Oct. 12,2003 
applause events shown (as infrequent as they might be) 
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FIG. 11b 
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Audio Feature Extraction 

Audio amplitude extraction and graphical 
approximation, An svg file is output. 
Music detection outputs time stamps, 



Document Format Specification 

Type = Radio program 
Identifier = 1774 

Title = A Prairie Home Companion 
Annotation = music events shown 
Publication Date = Sun, Oct. 12, 2003 
Begin time = 00:00:00 
End time = 01:58:56 

Graphical Representation = Amplitude curve 
2'-^ Feature extraction = music detection delta = 12.93 
Marker type = time stamp, bar code 
Marker placement = right of time fine 
Marker frequency = user-defined 

Marker 1 = (00:21:54, barcode, horiz. pos. 1} 

Marker 2 = (01 : 10:53, bar code, horiz. pos. 1} 

Marker 3 = (01 :34:01 , bar code, horiz. pos. 1 ) 

Marker 4 = (01 :41 :41 , bar code, horiz. pos. 1 ) 
2 Layout type = Two vertical time lines, split in half 
Layout pages = 1 

Layout marker placement = To right of graphical representation, as provided 
Layout meta data placement = Centered at top of page 
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A Prairie Home Companion 

Sun. Oct. 12,2 
music events shown 
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FIG. 12b 



